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Our  work  on  perception-based  planning  and  execution  continues: 

•  In  this  quarter,  we  extended  our  visual  servoing  capabilities  to  include  operations  such  as  visual 
alignment  along  an  axis,  and  full  six-degree-of-freedom  relative  positioning.  We  demonstrated  the 
use  of  alignment  by  programming  the  system  to  place  a  screwdriver  onto  a  screw.  As  with  all 
other  visual  control  operations,  these  are  calibration-insensitive.  We  also  demonstrated  vision- 
based  robot  control.  We  have  developed  a  small  piloting  program  that  permits  a  user  to  guide  the 
robot  using  visual  tracking.  The  user  can  point  at  objects  such  as  a  door  or  window,  or  simple 
features  such  as  corners  or  other  areas  with  high  contrast,  and  instruct  the  robot  to  home  on  those 
features  while  performing  obstacle  avoidance.  We  have  also  demonstrated  some  early  results  on 
selecting  features  to  track  automatically. 

•  We  have  been  running  experiments  to  test  our  object-recognition  algorithms.  We  generated  75 
images  by  dropping  two-dimensional  objects  into  random  cluttered  arrangements  on  a  table  top. 
Fifty  of  the  images  contained  the  target,  almost  always  occluded;  25  did  not  contain  it.  We  ran 
the  algorithm  on  all  75  images.  When  the  object  was  present,  the  algorithm  produced  an  average 
of  2.16  feasible  interpretations,  which  included  the  actual  object  whenever  it  was  present  and  less 
than  90%  occluded.  When  the  object  was  absent,  the  algorithm  produced  less  than  one  feasible 
interpretation  on  average.  The  goal  of  the  algorithm  is  to  filter  the  edge  sets  from  the  raw  image 
so  that  a  detailed  matcher  has  to  be  called  only  once  or  twice  per  image.  So  far,  it  appears  to 
be  completely  successful.  The  bogus  interpretations  the  algorithm  finds  can  be  quickly  rejected 
by  slightly  more  sophisticated  matching  algorithms.  The  results  will  be  reported  in  forthcoming 
paper  by  Tagare  and  McDermott. 

•  Our  planning  work  focused  on  the  redesign  of  the  XFRM-ML  notation  for  expressing  plan  tians- 
formations;  and  on  reorganizing  our  declarative  planner.  XFRM-ML  a  PROLOG-like  language  that 
comprises  PROLOG  primitives,  a  LISP  interface,  and  a  set  of  built-in  predicates  for  temporal  reason¬ 
ing,  as  well  as  predicates  on  task  networks  and  plans.  These  built-in  predicates  reconstruct  when 
necessary — the  state  of  the  robot  and  its  environment,  the  status  of  tasks  (succeeded,  failed,  active, 
etc.),  and  the  value  of  program  variables  and  data  structures  at  arbitrary  points  in  a  projected 
scenario. 

Using  XFRM-ML,  we  can  characterize  important  kinds  of  execution  failures  of  robot  plans  that 
cannot  be  represented  in  other  planning  representations.  XFRM-ML  also  allows  for  concisely 
coding  plan  revision  methods  for  reactive  plans  with  complex  control  structures.  We  gain  the 
necessary  expressiveness  of  XFRM-ML  by  not  restricting  the  representation  to  the  predicted  effects 
of  plan  execution  on  the  world,  but  in  addition  representing  how  the  plan  interpretation  changes 
the  state  of  the  agent,  and  how  plan  interpretation  is  connected  to  change  in  the  environment.  Just 
predicting  the  effects  of  the  plan  does  not  suffice  to  diagnose  plan  failures  caused  by  overlooking 
objects  or  faulty  world  models.  Based  on  the  XFRM-ML  representation,  our  planner  can  infer 
answers  to  questions  like,  has  a  particular  subplan  been  executed?,  why  (not)?,  etc. 

Activities: 

Greg  Hager  organized  a  workshop  on  Visual  servoing  at  the  International  Conference  on  Robotics 
and  Automation,  in  May 

Greg  Hager  gave  a  talk  at  the  DLR  Oberpfaffenhoffen  in  Germany 

Drew  McDermott  gave  an  invited  talk  entitled,  “The  Other  Problem  with  Classical  Planning,'’  at 
the  Second  International  Conf.  on  AI  Planning  Systems,  Chicago,  June  13 
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